Russ Allbery: rra-c-util 8.2
This release of my general utility libraries and support code includes a
large grab bag of fixes and improvements.
portable/system.h now defines explicit_bzero in terms of
memset if it is not available. The memset version is
unlikely to have the same security properties since the compiler may
optimize it away, but that allows me to use explicit_bzero to erase
security data where it is available.
For packages with Kerberos tests, generating a test krb5.conf file
now works properly even if the system krb5.conf file does not set a
default realm, and a krb5.conf file dropped into the test
configuration directory now works properly. Thanks to Jeffrey Hutzelman
for the latter fix.
For packages with PAM modules, the ENTRY and EXIT logging
macros can now be used like function calls, and portable/pam.h now
defines PAM_MAX_RESP_SIZE if it isn't defined.
Header ordering in some of the portability socket code has been restored
to compatibility with a few ancient UNIX systems. This was accidentally
broken by the clang-format reformatting. Thanks to Julien LIE for the
fix.
A few bugs in the test for SPDX license identifiers have been fixed.
Finally, this release fixes warnings with Clang 10 and GCC 10.
You can get the latest release from the rra-c-util distribution page.
Earlier this year, I was supposed to participate to 
If you never heard of the 10x engineer myth, it's a pretty great concept. It boils down to the idea where an engineer could be 10x more efficient than a random engineer. I find this fantastically twisted.Last week, I sat and chat with Alexis Monville in 
It has been close to a year now that I've incorporated my new company, 
Example of rule matching returned in GitHub checks
Some people like to make bots talk to each other.
GitHub service status is not always green.
Here is my monthly update covering what I have been doing in the free software world during July 2017 (
The idea is simply to have one coherent place with pointers to all the stuff we have and provide, without repeating nor replacing other documentation.
My monthly report covers a large part of what I have been doing in the free software world. I write it for 
The Debian Handbook
I worked with Ryuunosuke Ayanokouzi to prepare a paperback version of the Japanese translation of my book. Thanks to the efforts of everybody, 
Julien Ries anthropological research showed us that, already more than two million years ago, Homo Habilis looks like Symbolicus, with aesthetic sensibility, sense of symmetry and consciousness of creativity. Gilbert Durand confirms that the specific activity of man, the identity card of Homo Sapiens, is the symbolic activity, an essential part of his creativity. Then, man is creative at the moment when his first activates his imaginative feature.
So we can ask ourselves, how did we miss the creativity of man, of which so much we feel the need, or at least where is it hiding now? But above all which kind of creativity are we talking about?
<
The diagram above shows how data are organized inside Gnocchi. Until version
1.3, there would have been only one file for each aggregation methods.
In the upcoming 2.0 version, Gnocchi will split all these data into smaller
parts, where each data split is stored in a file/object. This allows to
manipulate smaller pieces of data and to increase the parallelism of the CRUD
operations on the back-end leading to large speed improvement.
In order to split timeseries into several chunks, Gnocchi defines a maximum
number of N points to keep per chunk, to limit their maximum size. It then
defines a hash function that produces a non-unique key for any timestamp. It
makes it easy to find in which chunk any timestamp should be stored or
retrieved.
Data compression
Up to Gnocchi 1.3, the data stored for each metric is simply serialized using
The XOR algorithm implemented in Python is pretty slow, compared to LZ4. Truth
is that 
I tried 2 Python implementations of the same algorithm (and submitted them to
my friend and Python developer
Considering the compression ratio of the different algorithms, they are pretty
much identical. The worst case scenario (random values) for LZ4 compress down
to 9 bytes per data point, whereas XOR can go down to 7.38 bytes per data
point. In general XOR encoding beats LZ4 by 15 %, except for cases where all
values are 0 or 1. However, LZ4 is faster than XOR by a factor of 4 -70
depending on cases.
That means that we'll use LZ4 for data compression in Gnocchi 2.0. It's
possible that we could achieve as fast compression/decompression algorithm, but
I don't think it's worth the effort right now it'd represent a lot of code to
write and to maintain.